Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add writeBinary to ostream for PCDWriter #5598

Merged
merged 7 commits into from
Feb 14, 2023

Conversation

ptrckschcknbch
Copy link
Contributor

Add method writeBinary to serialize point clouds in memory uncompressed.

Solves #5539

Add method writeBinary to serialize point clouds in memory uncompressed.
@mvieth
Copy link
Member

mvieth commented Feb 7, 2023

Hi, thanks for the PR. I think, in the interest of keeping the existing writeBinary function as fast as possible, it would be better if the function with the filename does not call the function with the ostream, even if we can save a few lines by doing so. Luckily, the function with the ostream is quite short anyway since it does not need all the error checking and platform specific behaviour.
Regarding testing: you could adapt the test LZFInMem in test/io/test_io.cpp.
What do you think?

Add method writeBinary to serialize point clouds in memory uncompressed.
@ptrckschcknbch
Copy link
Contributor Author

Hi, thanks for the PR. I think, in the interest of keeping the existing writeBinary function as fast as possible, it would be better if the function with the filename does not call the function with the ostream, even if we can save a few lines by doing so. Luckily, the function with the ostream is quite short anyway since it does not need all the error checking and platform specific behaviour. Regarding testing: you could adapt the test LZFInMem in test/io/test_io.cpp. What do you think?

Yes, I see your point and adapted the code accordingly.

Regarding testing: I copied and adapted the test LZFInMem. However, this is currently failing. But I'm not sure if the failure is related to my changes. As far as I could see, the generateHeaderBinary function, which is called in the new function (and also in the existing writeBinary function), seems to generate a header, which is not accepted by the readHeader function used in the test. Apparently the logic to create fake fields in generateHeaderBinary expects cloud.fields to be sorted by their offset. In my case, field rgb comes before the normal vector, but has a bigger offset, which leads to a negative COUNT value in the header.

I guess I need to dig a bit deeper here to understand what's going on. Maybe you have some ideas?

@ptrckschcknbch
Copy link
Contributor Author

ptrckschcknbch commented Feb 7, 2023

Hi, thanks for the PR. I think, in the interest of keeping the existing writeBinary function as fast as possible, it would be better if the function with the filename does not call the function with the ostream, even if we can save a few lines by doing so. Luckily, the function with the ostream is quite short anyway since it does not need all the error checking and platform specific behaviour. Regarding testing: you could adapt the test LZFInMem in test/io/test_io.cpp. What do you think?

Yes, I see your point and adapted the code accordingly.

Regarding testing: I copied and adapted the test LZFInMem. However, this is currently failing. But I'm not sure if the failure is related to my changes. As far as I could see, the generateHeaderBinary function, which is called in the new function (and also in the existing writeBinary function), seems to generate a header, which is not accepted by the readHeader function used in the test. Apparently the logic to create fake fields in generateHeaderBinary expects cloud.fields to be sorted by their offset. In my case, field rgb comes before the normal vector, but has a bigger offset, which leads to a negative COUNT value in the header.

I guess I need to dig a bit deeper here to understand what's going on. Maybe you have some ideas?

I guess following change in common/include/pcl/impl/point_types.hpp might fix this issue. But I have now clue, what other effects this change might have.

Maybe you can have a look if I'm on the right track here? You probably don't want to have such changes in this PR, but rather separated?

grafik

Fix the sequence order in POINT_CLOUD_REGISTER macros to match structs
@mvieth
Copy link
Member

mvieth commented Feb 9, 2023

Interesting, thanks for investigating. To my knowledge, this was never noticed before. Intuitively, I would agree that the fields should be ordered to match the order in the memory, but I would like to think about whether correcting it for the three point types can break something.
I would propose to sort the fields by their offset at the start of generateHeaderBinary (maybe copy cloud.fields, then use https://en.cppreference.com/w/cpp/algorithm/sort with a custom comparison lambda), and put all changes in point_types.hpp in a separate PR. Sorting makes sense anyway because people might store clouds with custom point types which do not follow the necessary field ordering.

Sort cloud.fields by their offset value to generate correct fake fields
@ptrckschcknbch ptrckschcknbch marked this pull request as ready for review February 10, 2023 16:11
Copy link
Member

@mvieth mvieth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thank you!

@mvieth mvieth added changelog: fix Meta-information for changelog generation changelog: new feature Meta-information for changelog generation labels Feb 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog: fix Meta-information for changelog generation changelog: new feature Meta-information for changelog generation module: io
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants